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THE  PROBLEM 

Identifying  Individuals,  before  they  are  actually  trained,  who  are 
most  lil.ely  to  succeed  in  flight  training  is  a  continuing  problem  in  naval 
aviation.  By  improving  aviation  personnel  selection  tests,  the  U.S.  Navy 
could  reduce  attriticr  and  associated  monetary  losses  and  expose  fewer 
flight  students  to  potentially  hazardous  training  situations.  Since  World 
War  II,  the  Navy  has  funded  efforts  to  improve  selection  tests,  which  screen 
individuals  desiring  entry  into  flight  training.  A  methodological  problem 
associated  with  the  development  of  candidate  tests  for  predictive  valida¬ 
tion  is:  How  do  we  design  a  test  that  measures  abilities  relevant  for 
inclusion  in  validation  studies? 

FINDINGS 

Random  Sampling  of  Domain  Variances  was  a  methodology  developed  as  a 
product  of  the  Augmentation  of  Human  Factors  Engineering  Technology  Efforts 
program.  This  methodology  provided  the  basis  for  developing  a  complex  visual 
information  task,  which  was  designed  to  require  many  fundamental  cognitive 
processing  demands  that  are  presumably  necessary  in  performing  aviation 
display  tasks.  Measures  from  this  complex  visual  task  were  significant 
predictors  against  three  criterion  measures  of  U.S.  Navy  primary  flight 
training  success:  pass/fail,  flight  grades,  and  composite  scores.  The 
number  of  correct  responses  to  the  visual  task  accounted  for  7.5%  of  the 
predicted  variance  associated  with  the  pass/fail  criterion.  A  prediction 
model  of  the  number  of  correct  responses  to  this  task  plus  the  Navy's 
current  Biographical  Inventory  score  accounted  for  10.2%  of  the  pass/ fail 
criterion  variance  in  a  sample  of  451  student  naval  aviators.  The  present 
results  indicate  that  this  compla.1:  visual  task  may  be  measuring  critical 
basic  cognitive  processes  that  mediate  both  general  intelligence  test 
performance  and  success  in  naval  aviation. 

RECOMMENDATIONS 

This  complex  visual  display  task  should  be  further  evaluated  in  a 
cross-validation  study  and  then  considered  for  implementation  as  a 
performance-based  selection  test.  The  method  of  Random  Sampling  of  Domain 
Variances  should  be  considered  in  the  design  of  future  tests  selected  for 
validation  studies. 
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INTRODUCTION 


This  effort  resulted  from  a  research  program  entitled  "Augmentation  of 
Human  Factors  Engineering  Technology  Efforts"  (AHFETE),  which  was  a  6-year 
Special  Focus  program  funded  by  the  Office  of  Naval  Research,  via  the  Naval 
Air  Systems  Command,  to  the  Naval  Aerospace  Medical  Research  Laboratory. 

The  program  began  in  1981  and  successfully  terminated  in  1986.  From  1983 
through  1986,  the  Naval  Air  Development  Center  and  the  Naval  Aerospace 
Medical  Research  Laboratory  participated  jointly  in  the  AHFETE  program. 

The  technical  objective  of  the  AHFETE  program  was  "To  develop  techni¬ 
ques  to  quantify  operator  workload  capacities  in  terms  of  systems  demands 
in  order  to  provide  common  quantitative  units  for  assessing  the  effects  of 
task,  environmental,  and  operator  variables  on  human  effectiveness  while 
performing  tasks  relevant  to  naval  aviation"  (1,  p.  1).  Due  to  the  ever 
increasing  array  of  instrumentation  that  aircrew  must  respond  ;o  and  the 
concomitant  increase  in  information  processing  demands,  the  need  to 
consider  human  capabilities  and  limitations  in  systems  design  is 
critical  (2,3).  Improved  assessment  of  these  human  capabilites  in 
aviation-relevant  tasks  could  provide  performance-bused  criteria  relevant 
to  the  design  of  emerging  systems. 

A  major  product  of  the  AHFETE  progra:  was  a  methodology,  Random 
Sampling  of  Domain  Variances,  to  sample  pe.'ormance  variance  estimates  that 
are  genera lizable  back  to  the  real-world  display  task  domain  (5,6).  Other 
major  products  were  the  development  and  the  application  of  hierarchical 
factor  analytic  techniques  to  identify  specific  perceptual  process?*  and 
their  processing  times  of  data  generated  herein  (7,8).  Morrison  (9)  pre¬ 
sented  a  less  elaborate,  basic  multiple  regression  approach  in  which  the 
regression  weights  associated  with  particular  display  task  cognitive  de¬ 
mands  were  interpreted  as  estimates  of  the  times  required  for  the  processes 
to  occur. 

The  primary  purpose  of  this  study  was  to  investigate  the  utility  of  a 
complex  visual  task  (CVT)  in  selecting  naval  aviatcra,  who  perform  many  of 
their  required  functions  via  various  complex  visual  displays.  The  visual 
dispin,,  task  selected  for  the  present  study  resulted  from  a  previous  func¬ 
tion/  task  analysis  of  the  P-3C  Tactical  Coordinator  (4).  A  critical  con¬ 
sideration  was  how  to  develop  the  experimental  display  task  in  a  way  to 
accurately  sample  the  real-world  display  task  domain. 

The  present  investigation  analyzed  mean  CVT  performance  cc^res.  Per¬ 
formance  scores  pertaining  to  individual  perceptual  processes,  which  may  be 
derived  via  the  hierarchical  factor  analytic  methods  previously  described 
(7,8),  were  not  analyzed.  If  CVT  measures  appear  useful  in  aviation  relec¬ 
tion,  then  the  CVT  test  may  be  included  in  other  selection  test  batteries 
currently  being  developed  c.ud  validated  at  the  Naval  Aerospace  Medical 
Research  Laboratory  (10,11).  A  secondary  purpose  was  to  examine  the  rela¬ 
tionship  between  CVT  measures  and  other  predictors  of  Navy  primary  flight 
training  success. 


METHOD 


SUBJECTS 

Student  naval  aviators  (441  males  and  10  females)  undergoing  Navy 
primary  flight  training  at  Naval  Air  Station  Whiting  Field,  Milton,  FL, 
participated  as  subjects.  All  subjects  had  passed  the  Navy  flight  physical 
examination.  Their  age  ranged  from  20  to  30  yr  old;  mean  age  was  23.2  yr. 

APPARATUS 

The  test  station  consisted  of  a  booth  in  which  the  seated  subject 
performed  the  complex  visual  task.  A  TV  monitor  and  Caramate  rear- 
projection  slide  system  were  positioned  in  front  of  the  subject,  with  the 
monitor  left  of  center  and  the  projector  right  of  center.  The  subject-to- 
screen  viewing  distance  was  about  51  cm.  A  response  keypad  with  keys 
labeled  0  through  9,  true,  false,  and  enter  was  positioned  directly  in 
front  of  the  subject.  The  keypad  contained  an  Apple  microcomputer,  inter¬ 
faced  to  a  switching  system,  that  controlled  task  presentation  and  recorded 
subject  response  times. 

One  hundred  and  twenty  slides  were  presented  on  an  illuminated  display 
screen.  The  screen  area  was  15.25  cm  x  15.25  cm,  which  was  divided  into 
quadrants  by  horizontal  and  vertical  lines  (each  had  a  1-mra  stroke  width). 

A  7.62-cm  diameter  circle  (1-mm  stroke  width)  was  centered  on  the  display 
screen.  Objects  presented  on  the  display  screen  varied  in  shape  (triaagle, 
rectangle,  pentagou);  color  (red,  green,  white);  size  (small  *  8  ram  L  x  6 
mm  W,  medium  =■  12  mm  L  x  8  mm  W,  large  »  15  mm  L  x  10  ram  W);  headiug  (N, 

NE,  E,  SE,  S,  SW,  W,  NW);  and  screen  location  (random).  The  number  of 
symbols  per  sl'de  varied  from  10  to  20.  The  dimensions  of  the  symbol  sizes 
were  measured  on  the  surface  of  the  display  screen.  Figure  1  illustrates 
the  display  format  and  the  three  shapes  and  three  sizes.  Each  symbol 
enclosed  a  solid  black  triangle,  which  indicated  symbol  heading  and  infer¬ 
red  "movement."  In  60  slides,  all  symbols  were  the  same  color;  in  13  and 
47  slides,  symbols  were  two  or  three  colors,  respectively.  For  all  cases, 
each  symbol  was  one  color.  Triangles  represented  airplanes;  rectangles  and 
pentagons  represented  aircraft  carriers  and  destroyers,  respectively.  The 
three  shapes  were  presented  in  all  three  sizes  and  all  three  colors. 

The  TV  monitor  presented  the  subject  with  questions  written  in  capital 
letters  7  mm  H  x  5  mm  W  and  stroke  width  approximately  1.5  mm.  Target 
characteristics  in  questions  to  identify  display  symbols  requiring  the  sub¬ 
ject's  response  are  given  in  Table  1.  Questions  differed  by  the  amount  and 
the  type  of  information  asked.  A  simple  question  was:  "How  many  red 
carriers  are  on  the  screen?"  This  question  required  subjects  to  memorize 
and  recall  two  types  of  target  symbol  information  (color  =  red,  shape  = 
carrier)  to  successfully  respond  to  the  subsequently  presented  display 
screen.  Although  the  question  included  the  words  "...  on  the  screen,"  it 
did  not  specify  a  particular  screen  portion,  hence,  the  subject  did  not 
have  to  remember  where  on  the  screen  (e.g.,  upper,  right,  left,  etc.)  to 
search.  A  aioi  j  difficult  question  was:  "At  least  two  3mall  red  destroyers 
heading  south  are  in  the  upper  screen  portion  (true  or  false)?"  This 
question  included  six  kinds  of  information  to  be  memorized  and  recalled: 

1)  number  of  question  objects  =  2;  2)  size  *  small;  3)  color  =  red;  4) 
shape  =  destroyer,  5)  heading  =  south;  and  6)  screen  portion  =  upper. 
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PROCEDURE 


Taped  verbal  instructions  (about  5  min)  and  programmed  example  slides 
were  presented  via  a  slide  projector  and  display  screen  to  each  subject, 
who  was  seated  in  the  test  booth..  The  experiment  followed  ttv'  instructions 
and  consisted  of  3  slide  group  presentations,  each  containing  40  slides. 

The  order  of  slide  presentation  within  a  slide  group  was  constant,  but  the 
slide  group  order  was  random  across  subjects.  The  experimenter  started  the 
first  trial  of  each  slide  group,  and  thereafter  the  experiment  was  self- 
paced.  A  trial  consisted  of  the  following:  1)  A  question  appeared  on  the 
TV  monitor;  2)  the  subject  read  the  question;  3)  the  subject  pressed 
"enter"  (reaction  time  1),  which  simultaneously  removed  the  question  from 
the  TV  monitor  and  presented  the  display  slide;  4)  the  subject  visually 
examined  the  display  slide  and  responded  via  the  keypad  (reaction  time  2) 
in  accordance  with  the  immediately  preceding  question;  5)  the  display  slide 
was  removed  from  view;  and  6)  feedback  was  presented  on  the  TV  monitor. 

The  next  trial  began  when  the  subject  pressed  "enter"  again.  Reaction 
times  1  and  2  were  measured  in  milliseconds.  Subjects  required  20-25  min 
to  complete  each  slide  group.  The  Statistical  Analysis  System  (SAS)  ver¬ 
sion  5.16  (12)  was  used  to  analyze  the  data. 


TABLE  1.  Target  Characteristics  of  Questions  Presented  on  the  TV  Monitor 
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Target 

characteristic 

Definition 

Circle  position 

Target  symbols  were  either  inside  or  outside  the 
circle. 

Screen  portion 

Target  symbols  could  be  in  the:  upper,  lower, 
right,  left,  upper-left,  upper-right,  lower-left, 
or  lower-right  position  of  the  display  screen.  No 
specified  screen  area  meant  targets  could  be  anywhere 
in  the  full  screen. 

"Movement"  to/ from 
circle 

Target  symbols  could  be  "moving"  to  or  away  from  the 
center  of  the  circle,  or  would  pass  through  or  enter 
the  circle.  "Movement"  was  inferred. 

Shape 

Triangle  =>  airplane;  rectangle  *  carrier;  pentagon  =■ 
destroyer. 

Color 

Red,  green,  white. 

Size 

Small,  medium,  large. 

Heading 

North,  East,  South,  West,  NE,  SE,  SW,  NW, 

Each  red  object 
represented  2  objects 

This  required  subject  to  count  each  red  object  as 
really  representing  two  objects. 

Assume  all  nsu  objects 
are  rotated  90  degrees 
to  tbs  right 

This  required  subjects  to  mentally  rotate  certain 
display  symbols. 

Number  of  display 
obj  ects 

Total  number  of  symbols  on  display  screen. 

Target  symbols 

Number  of  target  symbols  to  search  for  on  display 
screen. 

Number  of  question 
objects 

Number  of  target  symbols  specified  in  question. 

Two  shapes 

Certain  questions  included  two  shapes,  e.g., 
carriers  and  airplanes. 

Two  headings 

Certain  questions  called  for  target  objects  heading 
in  either  of  two  .specified  directions. 

Two  sizes 

Certain  questions  called  for  targets  of  two  sizes. 

Two  colors 

Certain  questions  called  for  targets  that  wei;e 
either  of  two  colors. 
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For  purposes  of  comparison,  currently  used  predictors  and  measures  of 
flight  training  performance  are  provided  in  the  following  two  sections. 

PREDICTORS  OF  FLIGHT  TRAINING  SUCCESS 

The  following  is  a  list  and  brief  description  of  screening  tests  currently 
used  by  the  U.S.  Navy: 

1.  The  Academic  Qualification  Test  (AQT)  is  a  60-min  test  of 
general  intelligence.  The  AQT-C  is  a  converted  AQT  score:  mean  =  5; 
standard  deviation  =  1.96;  range  -  1-9  (i.e.,  AQT-Cs  are  stanine  scores). 

2.  The  Flight  Aptitude  Rating  (FAR)  is  a  composite  score  based  on 
the  Spatial  Apperception  Test,  Mechanical  Comprehension  Test,  and  Biograph¬ 
ical  Inventory.  The  FAR-C  is  the  converted  stanine  FAR. 

3.  The  Spatial  Apperception  Test  (SAT)  is  a  10-min  test  of  the 
ability  to  orient  in  space  or,  specifically,  to  visualize  the  relationship 
between  th-a  attitude  of  an  airplane  and  the  territory  It  flies  over.  The 
SAT-C  is  a  r  '  erted  half-step  stanine  score:  mean  =  10;  standard  devia¬ 
tion  *  3.96  ge  »  1-19, 

A.  i  Mechanical  Comprehension  Test  (MCT)  is  a  40-min  test 
of  the  ability  to  perceive  physical  relationships  and  handle  familiar 
concepts  of  everyday  mechanics.  The  MCT-C  is  a  converted  halt-step  stanine 
score. 

5.  The  Biographical  Inventory  (31)  is  an  untimed  questionnaire 
of  personal  history,  expressions  of  interest  and  attitudes,  and  selected 
information  items.  The  BI-C  is  a  converted  half-step  stanine  score. 

6.  The  Officer  Aptitude  Rating  (OAR)  is  derived  by  combining 
scores  from  the  AQT  and  the  MCT.  The  OAR-C  is  a  converted  score,  i.e.,  a 
Navy  standard  score,  where  mean  =*  50;  standard  deviation  »  10;  range  »  20- 
BO.  The  AQT  and  the  FAR  are  used  to  screen  applicants  for  entry  into  naval 
flight  training.  The  OAF.  is  used  to  screen  individuals  in  the  process  of 
becoming  an  officer  in  the  U.S.  Navy.  If  an  individual  has  taken  the  OAR 
and  subsequently  seeks  entry  into  naval  flight  training,  then  the  AQT  and 
the  FAR  are  administered  (even  though  the  individual  has  already  taken  the 
AQT  and  the  MCT  as  the  OAR). 

PRIMARY  FLIGHT  TRAINING  MEASURES 

The  following  is  a  list  and  brief  description  of  grades/ scores  obtained 
by  individuals  undergoing  Navy  primary  flight  training: 

1.  Flight  Grade  (FG)  is  the  overall  average  flight  grade  at  the 
end  of  primary  flight  training.  It  is  bused  on  item  grades  from  various 
training  flights  as  follows:  unsatisfactory  =  1;  below  average  =  2; 
average  =  3;  and  above  average  =  4.  Item  grades  from  37  aircraft  flights 
and  16  training  simulator  "flights"  are  equally  weighted  and  averaged  to 
produce  the  FG. 

2.  Academic  Grade  (AG)  is  the  overall  academic  grade  achieved 
through  primary  flight  training.  The  AG  is  a  converted  Navy  standard  score. 


3.  Composite  Score  (CS)  is  a  combination  of  the  FG,  AX,  and  AG, 
which  are  weighted  70%,  15%,  and  15%,  respectively.  Although  the  FG  is  the 
most  important  factor  in  determining  pipeline  assignment  following  primary 
flight  training,  the  CS  may  be  used  as  a  "tie-breaker"  for  students  who 
have  obtained  equal  FGs. 

4.  Pass/Fail  (P/F)  is  a  dichotomous  criterion  measure  pertaining 
to  primary  flight  training  outcome. 

5.  Aviation  Indoctrination  (Al)  is  the  average  of  certain  aca¬ 
demic  courses.  For  officers  under  instruction,  the  AI  is  an  average  of 
three  courses:  aerodynamics,  engineering,  and  navigation.  For  aviation 
officer  candidates,  the  AI  is  an  average  of  the  above  three  courses  plus 
eight  other  courses,  such  as,  officer  leadership  quality,  U.S.  seapower, 
military  law,  et  cetera. 


RESULTS 

Tables  2  and  3  present  the  zero-order  correlations  between  the  predic¬ 
tors  and  criterion  measures.  For  individuals  who  passed  primary  flight 
training,  CS,  AI,  AG,  and  FG  scores  were  available.  Those  scores  were  not 
available  for  individuals  who  failed,  hence,  they  do  not  appear  in  Table  3. 
Similarly,  no  P/F  measure  appears  in  Table  2  (because  everyone  passed). 

The  number  of  complex  visual  task  (CVT)  slide  presentations  responded  to 
correctly  was  designated  as  CVT-NC.  The  CVT-RT1  was  the  time  required  to 
read  the  question;  CVT-RT2  was  the  time  to  respond  to  the  display.  The 
CVT-RTls  and  CVT-RT2s  were  computed  from  the  correct  responses. 

Table  2  presents  many  significant  zero-order  correlations;  all  are  in 
the  expected  direction.  Age  correlated  negatively  with  the  AQT,  OAR,  CS, 
AI,  and  FG,  and  CVT-NC.  Tho  number  of  prior  flight  hours  (PFLT)  was 
related  positively  to  the  BI  and  the  CS.  Individuals  with  more  PFLT 
obtained  higher  BI  scores,  which  was  not  surprising  ac  the  Bl  contains  many 
aviation  knowledge/interest  items.  The  AQT  was  directly  related  to  the 
FAR,  SAT,  MCT,  OAR,  CS,  AI,  AG,  and  CVT-NC  and  inversely  related  to  age, 
CVT-RT1,  and  CVT-RT2.  Subjects  with  higher  AQT  scores  performed  a  greater 
number  of  CVT  trials  correctly.  Since  the  CVT-RT1  is  the  time  required  to 
read  and  commit  to  short-term  memory  alphanumerica lly  presented  informa¬ 
tion,  its  inverse  correlation  with  the  AQT  was  expected.  The  correlations 
between  the  FAR  and  the  SAT,  MCT,  and  BI  were  expectedly  high  and  positive, 
since  the  FAR  is  a  composite  of  the  latter  three.  Likewise,  the  FAR  varied 
directly  with  the  AQT,  OAR,  CS,  AI,  AG,  FG,  and  CVT-NC  and  varied  inversely 
with  the  CVT-RT2.  The  high  correlation  between  the  FAR  and  the  OAR  is 
expected,  since,  the  MCT  is  a  part  of  each.  Subjects  with  better  FAR 
scores  responded  faster  to  the  CVT  display  slide  and  obtained  higher  over¬ 
all  CVT  accuracy. 

As  shown  in  Table  2,  the  SAT  varied  directly  with  the  MCT,  OAR,  CS,  AI, 
FG,  AQT,  FAR,  and  CVT-NC;  the  SAT  was  inversely  related  to  the  CVT-RTl  and 
the  CVT-RT2.  Subjects  with  higher  SAT  scores  read  the  CVT  questions  faster 
and  responded  faster  to  the  CVT  display,  at  a  higher  level  of  accuracy. 

The  MCT  varied  directly  with  the  BI,  OAR,  CS,  AI,  AG,  FG,  CVT-NC,  AQT,  FAR, 

and  SAT.  Higher  MCT  scores  were  associated  with  faster  responses  to 

the  CVT  display.  The  BI  was  directly  related  to  the  OAR,  CS,  AI,  AG,  FG, 
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PFLT,  FAR,  and  MCT.  The  BI  was  not  related  to  any  CVT  measures.  The  OAR 
varied  directly  with  the  CS,  AI,  AG,  FG,  CVT-NC,  AQT,  FAR,  SAT,  rlCT,  and 
BI.  Subjects  with  higher  OAR  scores  had  taster  CVT-RT2s  and  were  younger* 

The  high  positive  correlations  between  the  CS  and  the  AI,  AG,  and  FG 
were  expected,  as  the  CS  it  a  composite  of  the  latter  three  scores.  Higher 
CS  scores  were  associated  with  faster  CVT-RT1  and  CVT-RT2  response  times, 
and  a  greater  number  of  correct  responses  to  the  CVT.  Additionally,  the  CS 
varied  directly  with  the  PFLT,  AQT,  FAR,  SAT,  MCT,  BI,  and  OAR  and  varied 
inversely  with  age-  The  AI  scores  varied  directly  with  the  AG,  FG,  CVT-NC, 
AQT,  FAR,  SAT,  MCT,  31,  OAR,  and  CS  and  varied  inversely  with  age.  The  AG 
varied  directly  with  the  FG,  CVT-NC,  AQT,  FAR,  MCT,  BI,  OAR,  CS,  and  AI. 

The  FG  scores  varied  directly  with  the  CVT-NC,  FAR,  SAT,  MCT,  BI,  OAR,  CS, 
AI,  and  AG  and  varied  inversely  with  age,  CVT-RT1,  and  CVT-RT2. 

The  CVT-NC  score  va  led  directly  with  the  AG,  FG,  AQT,  FAR,  SAT,  MCT, 
OAR,  CS,  and  AI  and  varied  inversely  with  age.  The  CVT-RT1  measure  varied 
inversely  with  the  AQT,  SAT,  CS,  and  FG  and  varied  directly  with  the  CVT-NC 
and  the  CVT-RT2.  The  CVT-RT2  measure  varied  inversely  with  the  FG,  FAR, 
AQT,  SAT,  MCT,  OAR,  and  CS  and  varied  directly  with  the  CVT-RT1  and  the 
CVT-NC.  The  general  trend  for  the  CVT-RT1  and  the  CVT-RT2  was  that  those 
who  were  faster  reading  the  CVT  question  and  responding  to  the  CVT  display 
also  had  higher  predictor  scores  and  obtained  higher  criterion  scores. 
Greater  accuracy  on  the  CVT  varied  directly  with  longer  CVT  question  read¬ 
ing  times  and  longer  CVT  display  response  times.  The  high  positive  correl¬ 
ation  between  the  CVT-RTl  and  the  CVT-RT2  indicates  that  subjects  who  read 
the  CVT  questions  faster  also  responded  to  the  display  faster. 

The  patteru  of  zero-order  correlations  obtained  in  the  P/F  group 
presented  in  Table  3  was  similar  to  that  in  Table  2.  An  important  correla¬ 
t'd  an  in  Table  3  was  between  the  CVT-NC  and  the  P/F,  For  data  analysis 
purposes,  a  pass  was  coded  as  a  "1"  and  a  tail  as  "2,"  thus  the  negative 
correlation  between  the  CVT-NC  and  the  P/F  indicates  that  successful  com¬ 
pletion  of  primary  flight  training  was  associated  with  higher  accuracy  on 
the  CVT. 

In  summary,  the  important  findings  regarding  performance  on  CVT  and 
subseouent  performance  duriug  primary  flight  training  were:  Response 
accuracy  (CVT-NC)  varied  directly  with  the  CS,  AI,  AG,  FG,  and  P/F.  Both 
the  CVT-RTl  and  the  CVT-R72  were  inversely  related  to  the  CS  and  the  FG. 
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TABLE  2.  Correlation  Matrix  of  Data  from  Student  Naval  Aviators  Who  Passed 
Primary  Flight  Training. * 


_1 

2 

2 

4 

2 

6 

2 

& 

2 

i£ 

1. 

Age 

2. 

PFLT 

.090 

3. 

AQT-C 

-.174 

-.070 

4. 

FAR-C 

-.023 

.022 

.215 

5. 

SAT-C 

-.026 

-.088 

.195 

.590 

6. 

MCT-C 

-.029 

-.004 

.310 

.689 

.240 

7. 

BI-C 

-.030 

.149 

.053 

.676 

.074 

.360 

8. 

OAR-C 

-.137 

-.046 

.810 

.537 

.263 

.771 

.229 

9. 

CS 

-.119 

.099 

.199 

.230 

.158 

.249 

.191 

.289 

10. 

AI 

-.103 

.078 

.359 

.273 

.107 

.371 

.184 

.452 

.654 

11. 

AG 

-.082 

.059 

.325 

.151 

.068 

.216 

.135 

.333 

.713 

.675 

12. 

FG 

-.098 

.096 

.093 

.185 

.141 

.184 

.156 

.185 

.927 

.452 

13. 

CVT-NC 

-.110 

.022 

.337 

.183 

.211 

.235 

-.038 

.360 

.198 

.240 

1  . 

CVT-RT1 

-.006 

.015 

-.148 

-.036 

-.122 

.035 

-.007 

-.089 

-.104 

-.067 

1  4. 

CVT-RT2 

.010 

.032 

-.099 

-.151 

-.123 

-.154 

-.056 

-.168 

-.128 

-.044 

11 

11 

11 

12. 

FG 

.500 

13. 

CVT-NC 

.233 

.130 

14. 

CVT-RT1 

-.067 

-.098 

.132 

15. 

CVT-RT2 

- .  013 

-.132 

,231 

.500 

*r 

*  +/-  .098;  £  < 

.05;  df  -  405. 

TABLE  3.  Correlation  Matrix  of 
406  Who  Passed  and  45 

Data  from  Student 
Who  Failed.* 

Naval 

Aviators  including 

2 

2 

2 

4 

5 

b 

l 

8  2 

10 

1.  Age 

2.  PFLT 

.106 

3.  AQT-C 

-.156 

-.073 

4.  FAR-C 

-.005 

.018 

.230 

5.  SAT-C 

-.024 

-.075 

.197 

.586 

6.  MCT-C 

-.005 

-.033 

.324 

.686 

.225 

7.  BI-C 

-o020 

.156 

.059 

.683 

.076 

.354 

8.  OAR-C 

-.110 

-.067 

.812 

.546 

.254 

.778 

.234 

9.  P/F 

.080 

.013 

-.056 

-.149 

-.055 

-.116 

-.162 

-.107 

10.  CVT-NC 

-.096 

-.004 

.327 

.189 

.189 

.254 

-.014 

.366  -,274 

11.  CVT-RT1 

-.002 

.002 

-.146 

-.013 

-.114 

.043 

.023 

-.082  -.058 

.119 

12.  CVT-RT2 

.016 

.025 

-.100 

-.141 

-.120 

-.150 

-.041 

-.161  -.025 

.216 

n 

12.  CVT-RT2 

.501 

*r  *  +/-  .093;  p  <  .05;  df  =  450. 
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Table  4  preseats  zero-order  correlations  between  the  AQT,  MCT,  SAT,  and 
BI  obtained  in  1963  (13)  and  1988  (the  p  sent  study).  The  correlations 
between  the  AQT  and  the  MCT,  the  AQT  and  :e  BI,  the  MCT  and  the  SAT,  und 
the  SAT  and  the  BI  have  remained  quite  sti >le  over  the  past  25  years. 
Fisher's  z  coefficients  were  determined  for  the  correlations  between  the 
AQT  and  the  SAT,  and  the  MCT  and  the  BI,  and  the  difference  between  the  £ 
coefficients  for  1963  and  1988  was  tested  for  significance  (14).  The 
difference  of  the  correlations  between  the  AQT  and  the  SAT  from  1963  to 
1988  was  significant,  £  <  *005.  Similarly,  the  difference  of  the  correla¬ 
tions  between  the  BI  and  the  MCT  from  1963  to  1988  was  significant, 

£  <  .0001.  Thus,  during  the  past  25  years,  the  zero-order  correlation 
between  the  MCT  and  the  BI  has  increased  while  the  correlation  between  the 
AQT  and  the  SAT  has  decreased. 


TABLE  4.  Correlation  Matrix  of  AQT,  SAT,  MCT,  and  BI  Scores:  1963  and  1988.* 


1963  (£ 

=  587) 

1988  (N 

-  446) 

AQT 

MCT 

SAT  BI 

AQT 

MCT 

SAT  BI 

AQT 

MCT 

.362 

.317 

SAT 

.272 

.250 

.194 

.223 

BI 

.026 

.179 

.099 

.060 

.361 

.078 

*Shoenberger  et  al.  (13). 


Separate  stepwise  multiple  regression  (R)  analyses  were  performed  with 
the  three  criterion  measures:  P/F,  FG,  and  CS.  The  significance  level  for 
the  £  statistic  for  a  variable  to  be  added  to  the  model  was  .05.  After  a 
variable  was  added,  however,  the  stepwise  method  examined  all  the  variables 
already  included  in  the  model  and  deleted  any  variable  that  did  not  produce 
an  J?  statistic  significant  at  .05.  Table  5  presents  the  results  of  the 
regression  analysis  with  P/F  as  the  criterion.  The  predictor  variables 
available  for  model  entry  were  the  MCT,  BI,  CVT-NC,  and  OAR;  these  obtained 
significant  zero-order  correlations  with  the  P/F  as  shown  in  Table  3. 
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TABLE  5.  Summary  Results  of  Stepwise  Multiple  Regression  Analysis  with 
Pass/Fail  as  Criterion. 


Step  2  Bl-C  entered.  R**2  =  0.10236 

Adjusted  R**2  =  0.09835 
R  -  0.31994 


Source 

df 

Sum  of  squares 

Mean  square 

F 

P 

Regression 

2 

4.06451709 

2.03225854 

25.54 

0.0001 

Error 

448 

35.64279999 

0.07955982 

Total 

450 

39.70731707 

Step 

Variable 

entered  removed 

Pa rtial 
R**2 

Model 

R**2 

F 

£ 

1 

CVT-NC 

0.0748 

0.0748 

36.33 

0.0001 

2 

BI-C 

0.0275 

0.1024 

13.73 

0.0002 

The  first  variable  to  enter,  CVT-NC,  accounted  for  7.48%  of  the  P/F 
variance;  the  BI  entered  second,  accounting  for  2.75%.  The  two-predictor 
model  was  highly  significant,  accounting  for  a  total  of  10,24%  of  the  P/F 
variance  (adjusted  R**2  -  .0984).  Of  all  the  predictors — age,  PFLT,  AQT, 
FAR,  SAT,  MCT  BI,  OAR,  CVT-NC,  CVT-RT1,  and  CVT-RT2— the  two  that  accounted 
for  significant  amounts  of  the  P/F  variance  were  the  BI  and  the  CVT-NC. 

Note  in  Table  3  that  no  CVT  measure  attained  significant  zero-order  correl¬ 
ations  with  the  BI. 

The  results  from  the  stepwise  regression  analysis  with  the  FG  as  the 
criterion  are  presented  in  Table  6.  The  predictor  variables  that  had 
obtained  significant  zero-order  correlations  (see  Table  2)  were  available 
for  model  entry  and  were:  age,  SAT,  MCT,  BI,  OAR,  CVT-NC,  CVT-RT1,  and 
CVT-RT2.  Since  the  FAR  is  a  composite  of  the  SAT,  MCT,  and  BI,  it  was  not 
available  tor  entry. 
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TABLE  6.  Summary  Results  of  Stepwise  Regression  Analysis  with  Flight  Grade 
as  Criterion. 


Step  5  OAR-C  ramoved,.  R**2  ■  0.06825 

Adjusted  R**2  *  0.06130 
R  =  0.26126 


Source 

i£ 

Sum  of  squares 

Mean  square 

F 

j> 

Regression 

3 

0.03507393 

0.01169131 

9.82 

0.0001 

Error 

402 

0.47879800 

0.00119104 

Total 

405 

0.51387193 

Variable 

Partial 

Model 

Step 

entered  removed 

R**2 

R**2 

F 

£ 

1 

OAR-C 

0.0343 

0.0343 

14.34 

0.0002 

2 

BI-C 

0.0136 

0.0479 

5.78 

0.0167 

3 

CVT-RT2 

0.0101 

0.0580 

4.29 

0.0390 

4 

CVT-NC 

0.0151 

0.0730 

6.51 

0.0111 

5 

OAR-C 

0.0048 

0.0683 

2.07 

0.1509 

The  predictors  entered  the  regression  model  in  the  following  order:  j 

OAR,  BI,  CVT-RT2,  and  CVT-NC.  On  the  fifth  step,  the  OAR  was  removed;  the  .] 

remaining  three  variables  accounted  for  6.83%  of  the  FG  variance.  Because  ] 

the  OAR  was  removed  from  the  model,  another  stepwise  analysis  was  performed  • 

with  only  the  BI,  CVT-NC,  and  CVT-RT2  available  for  entry.  The  results  1 

from  Table  7  show  that  the  overall  model  was  significant;  R**2  accounted  j 

for  6,83%  of  the  FG  variance  (adjusted  R**2  »  .0613).  Of  the  6.83%  pre¬ 
dicted  variance,  the  BI  accounted  for  2.44%,  CVT-NC  accounted  for  1.86%,  .  j 

and  CVT-RT2  accounted  for  2.53%. 
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TABLE  7.  Summary  Results  of  Stepwise  Regression  Analysis  with  Plight  Grade 
as  Criterion. 


Step  3  CVT-RT2  entered.  R**2  =  0.06825 

Adjusted  R**2  =  0.06130 
R  -  0.26126 


Source 

df 

Sum  of  squares 

Mean  square 

F 

£ 

Regression 

3 

0.03507393 

0.01169131 

9.82 

0.0001 

Error 

402 

0.47879800 

0.00119104 

Total 

405 

0.51387193 

Step 

Variable 

entered  removed 

Partial 

R**2 

Model 

R**2 

F 

£ 

1 

BI-C 

0.0244 

0.0244 

10.09 

0.0016 

2 

CVT-NC 

0.0186 

0.0430 

7.83 

0.0054 

3 

CVT-RT2 

0.0253 

0.0683 

10.91 

0.0010 

Results  of  the  stepwise  regression  aualysis  with  the  CS  as  the  criter¬ 
ion  are  presented  in  Table  8.  The  predictors  that  obtained  significant 
zero-order  correlations  (see  Table  2)  and  consequently  were  available  for 
model  entry  were:  age,  PFLT,  AQT,  SAT,  MCT,  BI,  OAR,  CVT-NC,  CVT-RT1,  and 
CVT-RT2.  The  FAR  score  was  not  available  for  entry  because  it  is  a  compos¬ 
ite  of  the  SAT,  BI,  and  SAT,  which  were  available.  Although  the  OAR  is  a 
composite  of  the  AQT  and  the  MCT,  it  is  a  separate  administration,  there¬ 
fore,  the  OAR  as  well  as  the  AQT  and  the  MCT  were  available  for  entry.  The 
predictors  entered  in  the  following  order:  OAR,  BI,  CVT-NC,  and  CVT-RT2. 
The  OAR,  BI,  CVT-NC,  and  CVT-RT2  accounted  for  8.35%,  1.64%,  1.40%,  and 
1.50%,  respectively,  of  the  obtained  R**2  =  .1289  (adjusted  R**2  »  .1202). 


matiou  bits  contained  la  the  chunk.  Presumably,  individuals  possessing 
superior  capabilities  to  encode,  maintain,  and  manipulate  a  greater  amount 
o£  information  in  STM  faster,  might  succeed  at  a  higher  rate  in  flight 
training.  The  results  of  the  regression  analyses  presented  above  support 
this  assertion. 

Basic  cognitive  processing  ability  has  been  related  to  intelligence. 
Hunt  et  ai.  (21)  listed  the  following  basic  cognitive  processes,  collect¬ 
ively  referred  to  ar  "Current  Information  Processing"  (CIP):  processes  by 
which  people  detect  stimuli,  locate  them  in  time  and  space,  integrate 
sequences  of  stimuli  over  time  to  form  a  stable  percept,  transform  the 
percept  during  problem  solving,  coordinate  them  with  respect  to  stored 
representations  of  highly  cverlearned  codes,  and  abstract  information  from 
the  percept  and  place  it  in  Long-term  memory.  Hunt  et  al.  (21)  compared 
subjects  (college  freshmen)  having  high-versus-Low  verbal  ability  (measured 
by  the  Washington  Pre-College  Test)  and  found  that  high  verbals  displayed 
the  following  superior  CIP  abilities:  ability  "o  make  a  rapid  conversion 
from  a  physical  representation  to  a  conceptual  meaning;  ability  to  maintain 
information  in  short-term  memory  about  the  order  of  stimulus  presentation; 
and  ability  to  be  more  rapid  in  the  manipulation  of  data  in  short-term 
memory.  Hunt  et  al.  (21)  regarded  information  processing  capacity  as  a 
more  basic  mental  ability  than  the  composite  skill  of  verbal  aptitude. 
Performance  ou  verbal  aptitude  tests  is  largely  determined  by  what  a  person 
knows  and  not  just  how  rapidly  someone  can  perform  basic,  simple,  cognitive 
processes  such  as  the  CIPs  described  above. 

The  correlation  between  the  CVT-NC  and  the  AQT  scores  was  .337;  the 
CVT-NC  was  related  to  the  P/F  (jr  ■=*  -.274),  but  AQT  scores  were  not  related 
to  the  P/F.  This  finding  provides  support  for  selection  tests,  such  as  the 
CVT,  that  measure  abilities  to  rapidly  process  information,  manipulate  the 
information  in  STM,  make  comparisons  of  this  information  held  in  STM  with 
new  incoming  information,  and  make  decisions  based  on  such  comparisons. 

The  CVT  is  similar  in  processing  demands  to  the  CIP  tests  of  Hunt  et  al. 
(21).  All  subjects  tested  on  the  CVT  possessed  the  knowledge  requirements 
to  perform  the  task  (i.e.,  all  knew  concepts  such  as,  triangle,  pentagon, 
rectangle,  and  directions  N,  S,  E,  W,  etc.,  prior  to  testing).  Nonethe¬ 
less,  subjects  differed  in  their  ability  to  accurately  perform  the  CVT,  a 
task  that  required  many  basic  processes  of  the  CIP  tests  of  Hunt  et  al. 
(21).  In  the  fifth  experiment  by  Hunt  et  al.  (21),  a  statement  (e.g.,  * 
ABOVE  +)  was  presented,  subjects  pressed  a  button  when  they  comprehended 
the  statement,  then  a  picture  of  the  *  and  +  was  presented  in  various 
spatial  arrangements.  Subjects  respouded  true  or  false.  Statement  encod¬ 
ing  and  decision  times  (in  response  to  the  picture)  were  measured.  With 
negative  statements,  the  encoding  and  decision  times  for  high  verbals  were 
approximately  half  that  of  low  verbals.  For  the  CVT,  higher  accuracy  was 
directly  related  to  passing  primary  flight  training  and  higher  AQT  scores. 
The  present  results  indicate  that  the  CVT  may  be  measuring  critical  basic 
cognitive  processes  that  mediate  both  general  intelligence  test  performance 
and  success  in  naval  aviation.  Aside  from  predicting  success  in  naval 
aviation,  the  CVT  has  the  potential  to  measure  relevant,  basic  cognitive 
processing  capabilities  without  bias  against  population  minorities, 
assuming  that  all  college  graduates  possess  the  necessary  background  know¬ 
ledge  required  to  perform  fhe  CVT. 
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Based  on  data  from  451  student  naval  aviators,  the  CVT-NC  was  a  consis¬ 
tent  predictor  of  success  during  Navy  primary  flight  training.  This  mea¬ 
sure  of  performance  accuracy  on  the  CVT  predicted  tc  all  three  criterion 
measures  of  Navy  primary  flight  training  success:  F/F,  FG,  and  CS.  With 
the  BI  score  (a  test  already  administered  by  the  U.S.  Navy)  and  the  CVT-NC 
score,  10.24%  of  the  F/F  variance  was  accounted  for.  Given  that  the  BI  is 
already  available,  this  increase  In  prediction  capability  from  one  addi¬ 
tional  test  is  quite  significant.  These  findings  provided  a  critical 
external  validation  for  the  methodology  of  Random  Sampling  of  Domain  Vari¬ 
ances  (5),  which  was  developed  as  part  of  the  AHFETE  Program  and  to  which 
the  success  of  the  CVT  as  a  predictor  test  was,  no  doubt,  largely  due.  The 
present  analyses  provided  strong  support  for  using  the  Random  Sampling  of 
Domain  Variances  methodology  in  selection  research,  as  well  as  in  human 
factors  engineering  design  efforts. 

The  following  recommendations  are  offered: 

1.  The  CVT  should  be  further  evaluated  in  a  cross-validation 
study  and  theu  considered  for  implementation  as  a  performance-based  selec¬ 
tion  test. 
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2.  The  method  of  Random  Sampling  of  Domain  Variances  should  be 
used  to  design  future  selection  tests.  This  methodology  provided  strong 
internal  validity  for  incorporating  real-world  task  demands  into  the 
"laboratory"  test.  The  analyses  presented  herein  also  provided  strong 
external  validity  in  support  of  this  methodology. 

3.  The  data  included  in  this  report  should  be  analyzed  further-- 
vfa  the  hierarchical  factor  analysis  methods  (8),  also  developed  under  the 
AHFETE  program.  These  modifications  to  standard  hierarchical  factor  analy¬ 
tic  techniques  were  developed  for  the  specific  task  of  identifying  particu¬ 
lar,  individual  underlying  internal  processes  (7)  that  could  then  be  re¬ 
lated  to  specific  criterion  measures.  Such  follow-on  analyses  of  the 
present  data  should  allow  identification  of  basic  perceptual/cognitive 
processes  critical  to  naval  aviation. 
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